Information providers seek to provide information consumers with desired information in a manner which is fast, accurate, and efficient, and which makes productive use of available information resources. For example, such information consumers may include network users who wish to access, e.g., over a computer network, desired information which is stored in computer memory. In such scenarios, the information provider may be responsible for providing access to the desired information within the computer memory. In a specific example, the information provider may provide a search engine to the information consumers, e.g., network users. Thus, the network users may submit a search request for the desired information, whereupon the information provider may execute the search with respect to the information stored in the computer memory, and may thereafter provide the desired information to the requesting user.
Often, the user may wish to receive the desired information as quickly as possible. Meanwhile, the amount of information to be searched within the computer memory may be quite large, while the amount of available computer memory itself is nonetheless finite. For these and other reasons, it may be difficult to locate the information desired by the user within an acceptable timeframe, in a manner which optimizes use of the computer memory (and other computing resources of the information provider).
In order to facilitate retrieval of the desired information within an acceptable timeframe, the stored information may be indexed. In general, such indexing may include identifying and/or extracting discrete elements from within the stored information, and then storing each such discrete element along with a reference to the location within the stored information from which the corresponding discrete element was extracted. In a specific example, the stored information may include a plurality of text documents, in which case the documents may be indexed by extracting individual words therefrom, and thereafter storing each word together with a reference to the (location within the) document from which the word was obtained.
At a later time, as referenced above, the information provider may receive a request for desired information from a network user, where the request may include one or more of the words indexed in the manner just described. By comparing the thus-received word(s) with the previously-indexed words, the information provider may quickly determine a location of one or more documents which contain the indexed word. Then, the information provider may provide one or more of the located documents to the user in response to the user's request.
As referenced above, an amount of information to be stored may be very large, i.e., may require a large amount of physical computer memory to store all of the information. Although indexing the stored information advantageously provides faster and more accurate searching thereof, the index itself also must be stored, and thereby adds to the requirements for available memory resources. Moreover, even if sufficient computer memory resources are available to store all the desired information and associated index information, there may be insufficient computing resources to utilize the index and provide the desired information to the user within an acceptable timeframe. Consequently, it may be difficult for information providers and/or information consumers to achieve desired results using conventional indexing techniques.